Portable device and control method thereof

ABSTRACT

A portable device which is capable of communicating with an external device and a control method thereof are disclosed. A method for transmitting user input of a portable device includes detecting navigating input in a navigation mode for controlling an external device which is connected by a network, wherein the external device displays displayable content and the navigating input is for navigating the displayable content which includes at least one input box, transmitting a control signal corresponding to the detected navigating input to the external device, displaying an indicator which indicates capability of mode switching from the navigation mode to a user input mode, initiating the user input mode by displaying the input box which is extracted from the displayable content when user input for mode switching is detected, and transmitting user input which is received through the displayed input box.

This application claims the benefit of U.S. Provisional PatentApplication No. 61/606,468, filed on Mar. 4, 2012, which is herebyincorporated by reference as if fully set forth herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to portable devices, and moreparticularly, to a portable device which is capable of communicatingwith an external device and a control method thereof.

2. Discussion of the Related Art

When a user uses electronic devices, mobility has been an importantissue. Recently, a variety of portable devices having performancecorresponding to a desktop computer as well as cellular phones have beenon sale. A user can use electronic information while in motion as thedevices decrease in size and weight.

In addition to a basic function such as data transmission and reception,the portable devices perform various functions, thereby increasing usethereof. Especially, users use portable devices having high mobility tocontrol other electronic devices. To this end, technology for providinga convenient user interface is needed.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a portable devicewhich can more conveniently and accurately transmit and receive userinput and a method for transmitting and receiving user input.Especially, the present invention provides a convenient user interfaceto a user when a portable device receives user input and transmits userinput to an external device under a circumstance in which the portabledevice is connected to the external device.

In an aspect of the present invention, a method for transmitting userinput of a portable device includes detecting navigating input in anavigation mode for controlling an external device which is connected bya network, wherein the external device displays displayable content andthe navigating input is for navigating the displayable content whichincludes at least one input box, transmitting a control signalcorresponding to the detected navigating input to the external device,displaying an indicator which indicates capability of mode switchingfrom the navigation mode to a user input mode, initiating the user inputmode by displaying the input box which is extracted from the displayablecontent when user input for mode switching is detected, and transmittinguser input which is received through the displayed input box.

The method may further include terminating the user input mode when theuser input for the displayed input box is transmitted, and switchingback to the navigation mode.

The user input for mode switching may include touch screen input, stylusinput, hardware button input, gesture recognition, image recognition andvoice recognition.

The method may further include displaying terminating interface forterminating the user input mode, and terminating the user input modewhen user input to the terminating interface is detected.

In the method, initiating the user input mode may further includedetermining priority of the each input box, wherein the priority dependson the position of the input box in the displayable content, anddisplaying the input box sequentially corresponding to the determinedpriority.

The priority further may depend on whether the input box is displayed onthe external device.

The displayed input box may be extracted from the displayed area of thedisplayable content.

The displayed input box may further be extracted from non-displayed areaof the displayable content.

In the method, initiating the user input mode may further includedisplaying a virtual keyboard with the input box as input means.

In the method, transmitting user input may further include displayingtransmitting interface for transmitting user input which is receivedthrough plurality of input box groups, wherein each input box groupincludes related input boxes and the input box groups are mutuallyindependent, and transmitting the user input which is received throughthe plurality of input box groups when user input to the transmittinginterface is detected.

In another aspect of the present invention, a device for transmittinguser input includes a sensor unit configured to detect user input, acommunications unit configured to communicate with an external device,and a controller in control of the above units, configured to performactions including detecting navigating input in a navigation mode forcontrolling the external device which is connected by a network, whereinthe external device displays displayable content and the navigatinginput is for navigating the displayable content which includes at leastone input box, transmitting a control signal corresponding to thedetected navigating input to the external device, displaying anindicator which indicates capability of mode switching from thenavigation mode to a user input mode, initiating the user input mode bydisplaying the input box which is extracted from the displayable contentwhen user input for mode switching is detected, and transmitting userinput which is received through the displayed input box.

The actions may further include terminating the user input mode when theuser input for the displayed input box is transmitted, and switchingback to the navigation mode.

The user input for mode switching may include touch screen input, stylusinput, hardware button input, gesture recognition, image recognition andvoice recognition.

The actions may further include displaying terminating interface forterminating the user input mode, and terminating the user input modewhen user input to the terminating interface is detected.

In the device, initiating the user input mode may further includedetermining priority of the each input box, wherein the priority dependson the position of the input box in the displayable content, anddisplaying the input box sequentially corresponding to the determinedpriority.

The priority further may depend on whether the input box is displayed onthe external device.

The displayed input box may be extracted from the displayed area of thedisplayable content.

The displayed input box may further be extracted from non-displayed areaof the displayable content.

In the device, transmitting user input may further include displayingtransmitting interface for transmitting user input which is receivedthrough plurality of input box groups, wherein each input box groupincludes related input boxes and the input box groups are mutuallyindependent, and transmitting the user input which is received throughthe plurality of input box groups when user input to the transmittinginterface is detected.

In a further aspect of the present invention, a method for transmittinguser input of a portable device includes detecting navigating input in anavigation mode for controlling an external device which is connected bya network, wherein the external device displays displayable content andthe navigating input is for navigating the displayable content whichincludes at least one input box, transmitting a control signalcorresponding to the detected navigating input to the external device,switching from the navigation mode to a user input mode by displayingthe input box which is extracted from the displayable content when userinput for mode switching is detected, and transmitting user input whichis received through the displayed input box.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIGS. 1A and 1B are diagrams showing webpages displayed on an externaldevice according to an exemplary embodiment of the present invention.

FIG. 2 is a diagram showing a display data sharing operation between anexternal device and a portable device according to an exemplaryembodiment of the present invention.

FIGS. 3A and 3B are diagrams showing an input interface displayoperation of a portable device according to an exemplary embodiment ofthe present invention.

FIGS. 4A and 4B are diagrams showing an input interface displayoperation of a portable device according to an exemplary embodiment ofthe present invention.

FIGS. 5A-1 to 5E are diagrams showing examples of input interfacesaccording to an exemplary embodiment of the present invention.

FIGS. 6A and 6B are diagrams showing mode switching of a portable deviceaccording to an exemplary embodiment of the present invention.

FIGS. 7A and 7B are diagrams showing execution of a user input mode of aportable device according to an exemplary embodiment of the presentinvention.

FIGS. 8A to 8D are diagrams showing user input trigger in a user inputmode of a portable device according to an exemplary embodiment of thepresent invention.

FIG. 9 is a diagram showing a display screen of a user input modeaccording to an exemplary embodiment of the present invention.

FIG. 10 is a block diagram showing a portable device according to anexemplary embodiment of the present invention.

FIG. 11 is a flowchart showing a user input transmission/receptionmethod of a portable device according to an exemplary embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of thepresent invention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

FIGS. 1A and 1B are diagrams showing webpages displayed on an externaldevice according to an exemplary embodiment of the present invention.

The webpage displayed on the external device may include inputinterfaces for receiving user input. An input interface may be definedas an interface through which the device can receive user input such ascharacters, numerals, symbols, etc. or receive user selection such as amouse click, a touch input, a remote controller input, etc. For example,the input interface may include a text input box in which a user caninput specific text, a radio button which can select one of a pluralityof entities, a checkbox which can check selection, a selection buttonwhich can execute a specific command, and a select list box which canperform selective input among a plurality of lists. The input interfacemay further include information about the aforementioned input box orbuttons. For instance, the input interface may include informationcorresponding to the title of the text input box. Hereinafter, the inputinterface may be referred to as an input box and a detailed descriptionthereof will be given later.

In FIG. 1A, a display of the external device may display a ‘Search’ textinput box in which words to be searched are input, and ‘Username’ and‘Password’ text input boxes in which an ID and a password are input.

The input interfaces included on the webpage may be distributed in someseparate regions on the webpage rather than in a partial region. Forexample, the ‘Search’ text input box may be located in an upper part ofthe webpage and the ‘Username’ and ‘Password’, text input boxes may belocated in a right lower part of the webpage as shown in FIG. 1A. Thenit is inconvenient to input text because a user should move a pointerand then click a text input box in order to input text in thecorresponding text input box.

Further, since the display of the external device can display only alimited area, the content of a single webpage may not be simultaneouslydisplayed. Accordingly, a user should use an upper/lower or left/rightscroll bar in order to use the content of an area which is notdisplayed. That is, a situation may occur in which some input interfacesincluded in a corresponding webpage are not displayed and are displayedonly through an upper/lower or left/right scroll operation. Such a caseis shown in FIG. 1B.

As shown in FIG. 1B, a ‘Sign in’ input button and a ‘Stay signed in’checkbox related to the ‘Username’ and ‘Password’ text input boxes aredisplayed on the external device only after a user scrolls down thewebpage. Accordingly, in order to select a ‘Stay signed in’ option or tosign in after inputting information in the ‘Username’ and ‘Password’text input boxes, the case may occur in which a user should scroll downthe webpage and then click the ‘Stay signed in’ checkbox or the ‘Signin’ input button.

Moreover, if an input interface which has been displayed is moved to anarea which is not displayed on the external device due to a scroll-downoperation, a situation may occur in which the external device cannotreceive user input for the corresponding input interface. For example,in FIG. 1B, the ‘Search’ text input box is not displayed any longer on ascreen. Accordingly, the external device cannot receive an input in the‘Search’ text input box in the displayed webpage as shown in FIG. 1B anda screen scroll operation should be performed to receive an input in the‘Search’ text input box.

FIG. 2 is a diagram showing a display data sharing operation between anexternal device and a portable device according to an exemplaryembodiment of the present invention.

An external device 11 may transmit data related to a screen(hereinafter, referred to as screen data) which is being displayed to aportable device 13 which is connected thereto by wire or wirelessly. Theportable device 13 receives and displays the screen data on a display sothat displayable content displayed on the external device 11 can beprovided to a user through the display of the portable device 13.

The displayable content on the external device 11 includes contentusable online and content usable offline. For example, the displayablecontent may include Internet content and intranet content which areconnected online and include at least one input interface. In addition,the displayable content may include video content, multimedia content,etc. usable online or offline. Although content does not include aninput interface, if the input interface is included in a program orapplication for activating the corresponding content, the content may bedisplayable content. Hereinafter, a webpage will be described as anembodiment of the displayable content but the range of the presentinvention is not limited thereto.

The external device 11 refers to a device connected to the portabledevice 13 to transmit and receive data and may include a device having adisplay function, such as a television, a monitor, a projector, etc. Theexternal device 11 may also include another portable device connected tothe portable device 13.

As shown in FIG. 2, the portable device 13 may receive data for adisplay screen of the external device 11 and provide the received datato a user. The portable device 13 is well-equipped with an input toolfor user input and a user is also familiar with the input tool. Sincethe portable device 13 can be carried by a user, it is more frequentlyused than other external devices. Meanwhile, since user input for theexternal device is less used than the portable device and an input toolof the external device is limited to a remote controller etc., accuracyand input speed when such an input tool is used may be deteriorated.

Accordingly, it is convenient to use the portable device 13 if it isdesired to input specific information while surfing a webpage. Theportable device 13 may provide a user with an input interface on adisplay thereof through transmission and reception of screen data withthe external device 11. For example, as shown in FIG. 2, a user shouldinput a username and a password in text input boxes to log in to aspecific webpage. In this case, user input can be rapidly and accuratelyreceived through the portable device 13 by displaying input interfacesand an input tool rather than through the external device. The portabledevice 13 may provide a virtual keyboard as an input tool which iscapable of inputting text on the displayed input interfaces.

FIGS. 3A and 3B are diagrams showing an input interface displayoperation of a portable device according to an exemplary embodiment ofthe present invention.

The portable device 13 may display input interfaces by receiving data ofa displayed webpage from the external device 11. The portable device 13may display the input interfaces, through which a user can inputspecific text, among the content of a webpage. At this time, theportable device 13 may display both the input interfaces and informationcorresponding to titles of the input interfaces. In consideration of adisplay region of the portable device 13, the displayed input interfacesand related information may be enlarged at a given ratio.

FIG. 3A shows an embodiment of displaying input interfaces included in adisplay region of the external device on the portable device. Theportable device 13 may display input interfaces extracted from contentdisplayed on the external device of FIG. 3A. For example, the portabledevice 13 may display a ‘Search’ text input box for inputting text to besearched, a ‘Search’ input button for executing search, and ‘Username’and ‘Password’ text input boxes.

If a screen displayed on the external device 11 is changed, the portabledevice 13 may display input interfaces corresponding to the changedscreen. For example, if a user scrolls down the screen displayed on theexternal device so that input interfaces such as a ‘Sign in’ inputbutton and a ‘Stay signed in’ checkbox are displayed on the externaldevice as shown in FIG. 3B, the portable device 13 may additionallydisplay the above input interfaces.

The portable device 13 may also display input interfaces which are notdisplayed any longer on the external device 11 due to change of content.In other words, the portable device 13 may display input interfaceswhich do not appear on a display region of the external device 11 due toa screen scroll operation. Once the input interfaces are displayed onthe portable device 13, the portable device 13 may continue to displaythe input interfaces without deleting the displayed input interfaces.

As described with reference to FIG. 2, the portable device 13 maydisplay the input tool together with the input interfaces. Here, theportable device 13 may display a virtual keyboard as the input tool.

FIGS. 4A and 4B are diagrams showing an input interface displayoperation of a portable device according to an exemplary embodiment ofthe present invention.

Referring to FIG. 4A, the external device 11 may display a ‘Search’ textinput box, a ‘Search’ input button, and ‘Username’ and ‘Password’ textinput boxes. In FIG. 4B showing a result of scrolling down a screen ofthe external device 11, a ‘Sign in’ input button and a ‘Stay signed in’checkbox may be additionally displayed on the external device 11.

The portable device 13 according to the present invention may displayinput interfaces included in a webpage even before a screen of theexternal device 11 is scrolled.

In other words, even though input interfaces which are included in awebpage accessed by the external device 11 do not appear on a displayregion of the external device 11, the portable device 13 may display theinput interfaces on a display thereof. Accordingly, the portable device13 may provide input interfaces included in a webpage to a user withoutan additional control operation for display of the external device 11.

For example, even when the external device 11 displays some parts of awebpage as shown in FIG. 4A, the portable device 13 may display the‘Search’ text input box, ‘Search’ input button, ‘Username’ and‘Password’ text input boxes, ‘Sign in’ input button and ‘Stay signed in’checkbox included in the corresponding webpage. Through this, theportable device 13 may provide all input interfaces to a user without anadditional scroll operation for the webpage displayed on the externaldevice 11.

The portable device 13 may differentiate input interfaces displayed onthe external device 11 from input interfaces which are not displayed onthe external device 11 to differently configure regions of the inputinterfaces displayed on the portable device 13. Further, differentpriorities may be assigned to input interfaces according to theirlocations in displayable content displayed on the external device 11.The portable device 13 may assign a first priority to input interfaceswhich are displayed on the external device 11 and may assign a secondpriority to input interfaces which are not displayed on the externaldevice 11 so that the respective input interfaces have differentpriorities. According to such priorities, the portable device 13 maysequentially display the input interfaces.

For example, as shown in FIG. 4B, the portable device 13 may firstdisplay input interfaces which are displayed on the external device 11on a display of the portable device 13 and then display input interfaceswhich are not displayed on the external device 11.

As described with reference to FIG. 2, the portable device 13 maydisplay an input tool together with the input interfaces. Here, theportable device 13 may display a virtual keyboard as the input tool.

FIGS. 5A-1 to 5E are diagrams showing examples of input interfacesaccording to an exemplary embodiment of the present invention. FIGS.5A-1 shows text input boxes and information about the corresponding textinput boxes wherein the information is present outside the text inputboxes. The information about the text input boxes refers to informationexplaining the text input boxes and may include, for example, titleinformation of the text input boxes. Upon displaying extracted inputinterfaces, the portable device 13 may display only the text input boxesor may display both the text input boxes and the information about thetext input boxes.

FIGS. 5A-2 shows text input boxes and information about thecorresponding text input boxes wherein the information is present insidethe text input boxes. Upon displaying extracted input interfaces, theportable device 13 may display only the text input boxes or may displayboth the text input boxes and the information about the text inputboxes.

FIGS. 5A-3 shows an input interface including a plurality of text inputboxes. For example, when a telephone number or a credit card number isinput, a plurality of text input boxes connected with each other by adash may be used. The portable device 13 may display not only the textinput boxes but also information about the text input boxes and dashes.

FIG. 5B shows a select list box among input interfaces. The select listbox provides a list which can be selected in the box to a user when theuser clicks an arrow in the box, and the user may select a content to beinput from the list. Accordingly, the portable device 13 of the presentinvention may provide a list to a user by extracting and displaying thecontent of the list. Even in this case, information about the selectlist box may be displayed together with the select list box. Theinformation about the select list box may be the title of the selectlist box, for example, ‘Country’ in FIG. 5B.

FIG. 5C shows radio buttons among input interfaces. When one or moreselection items are present, a user can select an item to be input byclicking the radio button.

Upon extracting the radio buttons, the portable device 13 may extractand display information about the corresponding button. For example,text information ‘Yes’, ‘No’, ‘I Agree’, ‘I Disagree’ corresponding tothe titles of the radio buttons may be displayed together with the radiobuttons, as shown in FIG. 5C.

FIG. 5D shows checkboxes among input interfaces. A checkbox may indicatewhether a user selects corresponding content. The portable device 13 maydisplay only the checkboxes or may display information about thecheckboxes together with the checkboxes. The portable device 13 mayprovide the titles of the checkboxes, for example, ‘Stay signed in’ and‘Remember ID’ to a user together with the checkboxes.

FIG. 5E shows input buttons among input interfaces. If a user clicks aninput button, an execution command related to the input button may betransmitted. The button may include a text or an image indicatinginformation about the button. The portable device 13 may display acorresponding button and a text or image for the button. As shown inFIG. 5E, one of a plurality of input buttons may be selected. The inputbutton may include an input button for transmitting text input in a textinput box to a server managing a webpage. The ‘Search’ button and the‘Sign in’ input button shown in FIGS. 3A and 3B are examples of theinput buttons.

As described above, the portable device of the present invention maydisplay the text input boxes, checkboxes, radio buttons, and inputbuttons. Accordingly, in order to receive user input through inputinterfaces and transmit the user input to a server managing acorresponding webpage, the portable device may receive an executioncommand through an input button displayed thereon and may transmit inputtext, without transmitting a pointer shift control command for theexternal device.

FIGS. 6A and 6B are diagrams showing mode switching of a portable deviceaccording to an exemplary embodiment of the present invention.

The portable device may control a webpage region displayed on theexternal device through an upper/lower or left/right scroll operation ora pointer. This is called a navigation mode. If input interfaces areincluded in a webpage displayed on the external device, the portabledevice may be switched to a user input mode for receiving user input.Switching from the navigation mode to the user input mode may beimplemented by a trigger interface. A detailed description of thetrigger interface will be given with reference to FIGS. 8A to 8D.

The portable device 13 switched to the user input mode may display inputinterfaces extracted from a corresponding webpage. The respective inputinterfaces may be displayed together with related information and may beenlarged at a given ratio.

FIGS. 7A and 7B are diagrams showing execution of a user input mode of aportable device according to an exemplary embodiment of the presentinvention.

Although the portable device is switched from the navigation mode forthe external device to the user input mode, it is possible to beswitched from a standby mode to the user input mode as shown in FIG. 6.The standby mode refers to a state in which a background screen on whichapplication icons are listed is displayed rather than a state in which aspecific application is executed and displayed on the portable device.

If the external device displays a webpage on which input interfacesdemanding user input are present while a user surfs webpages, theportable device may be switched from the standby mode to the user inputmode so that the user can input text in the input interfaces. If a modeswitching command to the user input mode is received through a triggerinterface, the portable device is switched from the standby mode to theuser input mode to display input interfaces and receives user input.Accordingly, the portable device may directly be switched to the userinput mode from the standby mode even if it is not in the navigationmode for controlling the external device so that a user can input textin the input interfaces included in the webpage displayed on theexternal device.

Although the navigation mode and the standby mode has been described byway of example in FIGS. 6A, 6B, 7A, and 7B, the portable device of thepresent invention is not limited to switching from a specific mode tothe user input mode. The portable device 13 may be switched to the userinput mode even while executing an application.

FIGS. 8A to 8D are diagrams showing user input trigger in a user inputmode of a portable device according to an exemplary embodiment of thepresent invention.

As described with reference to FIGS. 6A to 7B, the portable device canbe switched to the user input mode from a navigation mode in which aremote control operation for a display of the external device isperformed or a scroll operation or a pointer shift operation for thecorresponding display is performed, a standby mode, or an applicationexecution screen. If a webpage displayed on the external device 11includes input interfaces, the portable device 13 may display anindicator indicating that the portable device can be switched to theuser input mode. In addition, the portable device may be switched to theuser input mode for receiving user input through input interfaces bydetecting user input for a mode switching. Hereinafter, user input formode switching may be referred to as a trigger interface and theindicator may be displayed independently of the trigger interface.

FIG. 8A shows an embodiment of an indicator indicating that the portabledevice can be switched to a user input mode, and a trigger interface,wherein the indicator and trigger interface are displayed on a touchscreen. In this embodiment, the indicator and the trigger interface mayuse the same screen region 81. The portable device 13 may provide theindicator and the trigger interface by displaying images or text whichare capable of receiving a touch input on a display thereof. If inputinterfaces are included in a webpage displayed on the external device,the portable device may display the screen region for mode switching ona partial or whole region of the touch screen. Upon detecting an inputon the screen region 81, the portable device 13 may be switched to theuser input mode to receive user input.

FIG. 8B shows another embodiment of the trigger interface. A stylus pen83 is used for the trigger interface. Upon receiving an input using thestylus pen 83, the portable device 13 may be switched to the user inputmode. The input using the stylus pen 83 may include touch for a specificpointer or behavior of drawing characters or shapes.

FIG. 8C shows still another embodiment of the trigger interface. Ahardware interface 85 is used for the trigger interface. For example,the portable device may be switched to the user input mode using ahardware button included therein. The hardware button may include amechanical button or an electronic button installed in the portabledevice. The portable device may be switched to the user input mode bysensing behavior of simultaneously pressing one or more buttons.

FIG. 8D shows still another embodiment of the trigger interface. Modeswitching is performed according to a location of a pointer 87 on awebpage displayed on the external device. If the pointer 87 is locatednear an input interface on the webpage displayed on the external device11, the portable device may be switched to the user input mode.

In addition to the above-described methods, the portable device may usegesture recognition using a sensor, voice recognition, and imagerecognition using a camera, as the trigger interface. Upon detecting aninput through the trigger interface, the portable device may be switchedto the user input mode.

FIG. 9 is a diagram showing a display screen of a user input modeaccording to an exemplary embodiment of the present invention.

The portable device may provide input interfaces to a user bysimultaneously displaying the input interfaces in a webpage displayed onthe external device. In this case, both input interfaces havingrelevance to each other and input interfaces having no relevance to eachother may be displayed. Here, input interfaces having relevance refer toinput interfaces having the same purpose.

As shown in FIG. 9A, since a ‘Search’ text input box and a ‘Search’input button have the same purpose for obtaining information forspecific search text, they may be defined as a first group 91 havingrelevance. In addition, ‘Username’ and ‘Password’ text input boxes, a‘Sign in’ input button, and a ‘Stay signed in’ checkbox which are usedfor a log-in purpose may be defined as a second group 93 havingrelevance. Namely, since the input interfaces of the first group 91 andthe input interface of the second group 93 have purposes for differentoperations, the first and second groups 91 and 93 have no relevance. Inthis case, a user may input text only in partial input interfaces ratherthan all input interfaces. Accordingly, when the ‘Search’ input buttonincluded in the first group is clicked, the portable device may transmituser input of the ‘Search’ text input box of the same group to theexternal device. Further, when the ‘Sign in’ input button included inthe second group is clicked, the portable device may transmit user inputof the ‘username’ and ‘password’ text input boxes and ‘Stay signed in’checkbox of the same group to the external device.

FIG. 9B shows the case where user input is received in input interfacesincluded in two or more groups having no relevance. As shown, if userinput is received in the input interfaces included in two or more groupshaving no relevance, the portable device may define these groups as onenew group 95. The portable device may additionally display atransmitting interface 99 and, upon receiving user input through thetransmitting interface 99, may transmit the received user input in theinput interfaces included in the new group 95 to the external device 11.The external device 11 may receive the user input and transmit the userinput to a server in which a webpage is executed.

If an input button included in each individual group is used, theportable device transmits only information received through inputinterfaces included in a corresponding group. Moreover, the portabledevice may additionally display the transmitting interface 99 so thatuser input through input interfaces having no relevance can besimultaneously transmitted.

Upon transmitting information received through input interfaces includedin two or more groups using the transmitting interface 99, the portabledevice may transmit information of a specific group first. For example,information of the first group 91 for search may be transmitted firstand then information of the second group 93 for log-in may betransmitted or vice versa. A transmitted order may be determined by theportable device or by a request of a corresponding webpage.

In the user input mode, the portable device may display a terminatinginterface 97 for ending the user input mode and returning to a previousmode in addition to the above-described transmitting interface 99. Theterminating interface 97 may be used when there is no user input afterthe user input mode is started or when a user desires to end the userinput mode. Upon detecting an input of the terminating interface 97, theportable device 13 may terminate the user input mode and return to aprevious mode. The previous mode may be the navigation mode, the standbymode, or an application execution screen described with reference toFIGS. 6A to 7B.

In addition to terminating the user input mode by receiving input forthe terminating interface 97, the portable device may terminate the userinput mode by receiving user input through an input interface and thenreceiving an execution command for an input button related to thecorresponding input interface or by receiving user input for thetransmitting interface 99 and then transmitting the user input. Asdescribed previously, the portable device 13 terminates the user inputmode and may return to the navigation mode, the standby mode, or anapplication execution screen.

FIG. 10 is a block diagram showing a portable device according to anexemplary embodiment of the present invention.

Referring to FIG. 10, the portable device includes a storage unit 101, acommunication unit 102, a sensor unit 103, an audio input/output (I/O)unit 104, a camera unit 105, a display unit 106, a power unit 107, aprocessor 108, and a controller 109.

The storage unit 101 may store a variety of digital data such as video,audio, pictures, and moving images. The storage unit 101 indicatesdigital data storage spaces such as a flash memory, a Hard Disk Drive(HDD), and a Solid State Drive (SSD). In the portable device of thepresent invention, the storage unit 101 is optional.

The communication unit 102 may transmit and receive data by performingcommunication using various protocols with an external network of theportable device. The communication unit 102 may transmit and receivedigital data by accessing the external network by wire or wirelessly.

The sensor unit 103 may transmit user input or an environment recognizedby the device to the controller 109 by using a plurality of sensorsinstalled in the portable device. The sensor unit 103 may include aplurality of sensing means. As an embodiment, the plurality of sensingmeans may include gravity sensors, geomagnetic sensors, motion sensors,gyro sensors, acceleration sensors, inclination sensors, brightnesssensors, altimeter sensors, odor sensors, temperature sensors, depthsensors, pressure sensors, bending sensors, audio sensors, videosensors, Global Positioning System (GPS) sensors, touch sensors, etc. Inthe present invention, the sensor unit 103 may sense user input forcontrolling the external device or user input for initiating the userinput mode or may receive user input to be input in input interfaces totransmit the user input to the controller 109. The sensor unit 103refers to the above various sensing means and may transmit a sensedresult by sensing a variety of user input and user environments so thatthe portable device can perform an operation according to the sensedresult. The above-mentioned sensors may be included in the portabledevice as separate elements or may be incorporated into at least oneelement.

The audio I/O unit 104 may include an audio output means such as aspeaker and an audio input means such as a microphone and may performaudio output of the device and audio input to the device. The audio I/Ounit 104 may be used as an audio sensor. In the portable device of thepresent invention, the audio I/O unit may be an optional element.

The camera unit 105 may capture pictures and moving images and may be anoptional element according to embodiments. The camera unit 105 may beused as a motion sensor or a visual sensor. In the present invention,the camera unit 105 may be used for image recognition which is one oftrigger interfaces for initiating the user input mode. Upon recognizinga specific image, the camera unit 105 may initiate the user input mode.Upon recognizing a specific image, the camera unit 105 may initiate theuser input mode. In the portable device of the present invention, thecamera unit 105 may be optional.

The display unit 106 may output images on a display screen. In the caseof a touch sensitive display, the display unit 106 may be used as theabove-mentioned touch sensor. The display unit 106 may detect user inputand transmit the detected user input to the controller 109. The displayunit 106 may display images on a display panel or control image display.In the present invention, the display unit 106 may display an inputinterface extracted from content and provide the input interface to auser. The display unit 106 includes a flexible display.

The power unit 107 is a power source connected to a battery in thedevice or to an external power and may supply power to the device. Inthe portable device of the present invention, the power unit may be anoptional element.

The processor 108 may execute various applications stored in the storageunit 101 and process data of the interior of the device.

The controller 109 may control the above-described units and managetransmission and reception of data between the units.

The processor 108 and the controller 109 may be incorporated into onechip 110 to perform the above operation. In this case, they may bereferred to as the controller 109.

In the present invention, the controller 109 may extract inputinterfaces from content. For example, the controller 109 may extractinput interfaces from content on a webpage and manage the display unit106 to display the extracted input interfaces.

In a block diagram according to an exemplary embodiment of the presentinvention shown in FIG. 10, separate blocks are shown by logicallydistinguishing between elements of the portable device. Theabove-described elements of the device may be configured as one chip orplural chips according to design of the device.

FIG. 11 is a flowchart showing a user input transmission/receptionmethod of a portable device according to an exemplary embodiment of thepresent invention.

The portable device detects user input for controlling an externaldevice connected through a network while using a navigation mode, astandby mode, or other applications as described in FIGS. 6A to 7B (stepS11). The input for controlling the external device may be variedaccording to content displayed on the external device.

The content which are displayable on the external device include contentusable online or offline. For example, the content may include awebpage, video content, multimedia content, intranet content, andInternet content which include at least one input interface connectedonline. The displayable content may also include video content andmultimedia content offline. Even when content do not include an inputinterface, if the input interface is included in a program orapplication for activating the corresponding content, the content may bethe displayable content.

In the case of the webpage, intranet content, and Internet content, theuser input may include a scroll operation of the corresponding webpageor content or a mouse pointer shift operation. The user input may alsoinclude a shift operation to the next or previous page. In the case ofthe video content and multimedia content, the user input may include anoperation of playing, stopping, fast forwarding, or rewinding video ormultimedia.

If the input interface is included within a program or application, theuser input may include an operation of searching specific words inInternet Explorer or searching information about content from amultimedia player.

Upon detecting the user input, the portable device transmits a controlsignal corresponding to the detected user input to the external device(step S13). The control signal may be transmitted through a networkconnecting the portable device with the external device and may betransmitted through the communication unit included in the portabledevice.

If the displayable content includes an input interface, the portabledevice may display an indicator indicating that a mode is switchable anda trigger interface for activating a user input mode as described withreference to FIGS. 8A to 8D (step S15). The trigger interface may beactivated when at least one input interface is included in a regiondisplayed on the external device. The trigger interface may be activatedeven when at least one input interface is included in a region exceptthe region in which a webpage or content are displayed although theinput interface is not included in the region displayed on the externaldevice. In other words, even though the input interface is not presenton a currently displayed screen region, if the input interface ispresent in a region which can be displayed by a scroll operation of thecorresponding screen, the trigger interface may be displayed even beforethe corresponding input interface is displayed by a scroll operation.

The step of activating the trigger interface may be performed such thatthe portable device displays the trigger interface when it is judgedthat an input interface is present as a result of analyzing, at theportable device, displayable content or such that the external deviceanalyzes content displayed thereon and informs the portable device ofinformation indicating the presence of the input interface and theportable device receives the information and displays the triggerinterface. Alternatively, while displaying the trigger interface, theportable device may activate the trigger interface only when the inputinterface is present, and otherwise, may inactivate the triggerinterfaces.

Upon sensing an input on the displayed trigger interface, the portabledevice displays at least one input interface extracted from displayablecontent to start a user input mode (step S17). The input interface isincluded in a webpage or content and an entity extracting the inputinterface may vary according to embodiments.

If the input interface is extracted by the portable device, the portabledevice analyzes data for displayable content received from the externaldevice and extracts and displays only the input interface from the data.Information about the input interface may also be displayed.

If the input interface is extracted by the external device, the externaldevice extracts data for the input interface included in the displayablecontent and transmits the data for the input interface to the portabledevice. The data about the input interface may further includeinformation about the input interface in addition to the inputinterface. The portable device may display the data for the inputinterface transmitted by the external device so as to be provided to auser

As described with reference to FIGS. 3A to 4B, the portable device maydisplay the input interfaces extracted from a screen region displayed onthe external device.

The portable device may determine priorities according to locations ofinput interfaces within content displayed on the external device.Namely, the portable device may assign different priorities to inputinterfaces displayed on a display of the external device and inputinterfaces which are not displayed. For example, the portable device mayfirst display the input interfaces displayed on the external device onthe display of the portable device and then may sequentially display theinput interfaces which are not displayed on the external device. Inaddition, the portable device may display a virtual keyboard as an inputtool together with the input interfaces.

Upon receiving an input through the displayed input interfaces, theportable device transmits the received input to the external device(step S19). As described with reference to FIGS. 9A and 9B, user inputfor input interfaces included in the same group are simultaneouslytransmitted. The received input is transmitted to the external device byclicking an input button displayed on a screen of the portable device orpressing an enter key after receiving input for an input interface. Iftwo or more groups are present, only an input of one of the two groupsmay be transmitted or inputs of the two groups may be sequentiallytransmitted.

The user input mode may be ended by the terminating interface. Theterminating interface may be displayed in the user input mode after theuser input mode is initiated in the portable device. The terminatinginterface may be used when there is no user input or the user input modeis ended without any user input. Upon detecting input for theterminating interface, the portable device terminates the user inputmode and returns to a previous state. The previous state may be anavigation mode, a standby mode, or an application execution screen.

The user input mode may also be terminated when the portable devicereceives input for an input interface and then clicking an input buttonrelated to the corresponding input interface or when the portable devicetransmits received input to the external device by detecting input forthe transmitting interface.

According to the present invention, only an input interface included incontent displayed on the external device can be selectively displayed onthe portable device and can be provided to a user.

According to the present invention, an input interface displayed on theportable device can receive user input and the received user input canbe transmitted to the external device.

According to the present invention, an input interface can be enlargedto be displayed on the portable device, so that accurate user input canbe provided to users having poor sight.

According to the present invention, an unnecessary pointer shiftoperation used for an input interface in content displayed on theexternal device is minimized and user input can be convenientlyprovided.

According to the present invention, user input received through aplurality of input interfaces can be simultaneously transmitted to theexternal device using an additional transmitting interface.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present invention. Thus,although the foregoing description has been described with reference tospecific examples and embodiments, these are not intended to beexhaustive or to limit the invention to only those examples andembodiments specifically described.

As described above, the present invention is totally or partiallyapplicable to electronic devices.

1. A method for transmitting user input of a portable device,comprising: detecting navigating input in a navigation mode forcontrolling an external device which is connected by a network, whereinthe external device displays displayable content and the navigatinginput is for navigating the displayable content which includes at leastone input box; transmitting a control signal corresponding to thedetected navigating input to the external device; displaying anindicator which indicates capability of mode switching from thenavigation mode to a user input mode; initiating the user input mode bydisplaying the input box which is extracted from the displayable contentwhen user input for mode switching is detected; and transmitting userinput which is received through the displayed input box.
 2. The methodof claim 1, further comprising: terminating the user input mode when theuser input for the displayed input box is transmitted; and switchingback to the navigation mode.
 3. The method of claim 1, wherein the userinput for mode switching includes touch screen input, stylus input,hardware button input, gesture recognition, image recognition and voicerecognition.
 4. The method of claim 1, further comprising: displayingterminating interface for terminating the user input mode; andterminating the user input mode when user input to the terminatinginterface is detected.
 5. The method of claim 1, wherein initiating theuser input mode further comprises: determining priority of the eachinput box, wherein the priority depends on the position of the input boxin the displayable content; and displaying the input box sequentiallycorresponding to the determined priority.
 6. The method of claim 5,wherein the priority further depends on whether the input box isdisplayed on the external device.
 7. The method of claim 1, wherein thedisplayed input box is extracted from the displayed area of thedisplayable content.
 8. The method of claim 7, wherein the displayedinput box is further extracted from non-displayed area of thedisplayable content.
 9. The method of claim 1, wherein initiating theuser input mode further comprises displaying a virtual keyboard with theinput box as input means.
 10. The method of claim 1, whereintransmitting user input further comprises: displaying transmittinginterface for transmitting user input which is received throughplurality of input box groups, wherein each input box group includesrelated input boxes and the input box groups are mutually independent;and transmitting the user input which is received through the pluralityof input box groups when user input to the transmitting interface isdetected.
 11. A device for transmitting user input, the devicecomprising: a sensor unit configured to detect user input; acommunications unit configured to communicate with an external device;and a controller in control of the above units, configured to performactions including: detecting navigating input in a navigation mode forcontrolling the external device which is connected by a network, whereinthe external device displays displayable content and the navigatinginput is for navigating the displayable content which includes at leastone input box; transmitting a control signal corresponding to thedetected navigating input to the external device; displaying anindicator which indicates capability of mode switching from thenavigation mode to a user input mode; initiating the user input mode bydisplaying the input box which is extracted from the displayable contentwhen user input for mode switching is detected; and transmitting userinput which is received through the displayed input box.
 12. The deviceof claim 11, the actions further comprising: terminating the user inputmode when the user input for the displayed input box is transmitted; andswitching back to the navigation mode.
 13. The device of claim 11,wherein the user input for mode switching includes touch screen input,stylus input, hardware button input, gesture recognition, imagerecognition and voice recognition.
 14. The device of claim 11, theactions further comprising: displaying terminating interface forterminating the user input mode; and terminating the user input modewhen user input to the terminating interface is detected.
 15. The deviceof claim 11, wherein initiating the user input mode further comprises:determining priority of the each input box, wherein the priority dependson the position of the input box in the displayable content; anddisplaying the input box sequentially corresponding to the determinedpriority.
 16. The device of claim 15, wherein the priority furtherdepends on whether the input box is displayed on the external device.17. The device of claim 11, wherein the displayed input box is extractedfrom the displayed area of the displayable content.
 18. The device ofclaim 17, wherein the displayed input box is further extracted fromnon-displayed area of the displayable content.
 19. The device of claim11, wherein transmitting user input further comprises: displayingtransmitting interface for transmitting user input which is receivedthrough plurality of input box groups, wherein each input box groupincludes related input boxes and the input box groups are mutuallyindependent; and transmitting the user input which is received throughthe plurality of input box groups when user input to the transmittinginterface is detected.
 20. A method for transmitting user input of aportable device, comprising: detecting navigating input in a navigationmode for controlling an external device which is connected by a network,wherein the external device displays displayable content and thenavigating input is for navigating the displayable content whichincludes at least one input box; transmitting a control signalcorresponding to the detected navigating input to the external device;switching from the navigation mode to a user input mode by displayingthe input box which is extracted from the displayable content when userinput for mode switching is detected; and transmitting user input whichis received through the displayed input box.